1. Field of the Invention
The present invention relates to a solid-state image pickup apparatus and a signal processing method therefor. This invention is advantageously applicable to, e.g., a digital camera, an image input unit or an image processing unit. The image pickup apparatus includes an image sensor constituting an input device and advantageously implemented as the image capturing section of, e.g., a digital camera, an image input unit or an image processing unit. Signals or images are processed, taking account of an arrangement of color filter segments of a color filter. This processing is, for example, desirably practicable with the image pickup apparatus.
2. Description of the Background Art
An interlacing type of solid-state image pickup apparatus is known in the art that includes a sensor array arranged in a honeycomb pattern. Specifically, the sensor array has photosensitive cells or photoelectric transducers (pixels) arranged bidimensionally. Nearby photosensitive cells are shifted from each other by one-half of a pixel pitch in the horizontal and vertical directions. More specifically, pixels on odd-numbered rows and pixels on even-numbered rows are shifted from each other by half a pitch in the direction of rows. Likewise, pixels on odd-numbered columns and pixels on even-numbered columns are shifted from each other by half a pitch in the direction of columns. The individual pixel has a tetragonal, hexagonal, octagonal or similar polygonal shape.
Vertical transfer paths extend zigzag in such a manner as to skirt round the pixels. Signal charges generated in the photosensitive cells are vertically transferred by, e.g., four-phase drive. The signal charges are then transferred to a horizontal transfer path via the vertical transfer paths and transferred along the horizontal path by, e.g., two-phase drive.
A color filter has color filter segments arranged in a pattern in front of the photosensitive cells in one-to-one correspondence. For example, the color filter segments are arranged in an R (red), G (green), B (blue) Bayer pattern inclined by 45 degrees.
The photosensitive cells having the above configuration directly output color signals of three primary colors R, G and B. Such color signals have a desirable S/N (Signal-to-Noise) ratio and high color reproducibility. The primary color filter, however, transmits only about one-third of wavelength components contained in incident white light, failing to efficiently use incident light. This limits the sensitivity and resolution of the resulting image.
To improve resolution, it has been customary to interpolate luminance data by using correlation of pixel data in the direction of rows and the direction of columns. Also, for the efficient use of incident light, it is a common practice to use color filter segments of complementary colors. Complementary color filter segments can use incident light with efficiency about two times as high as the efficiency available with the primary color filter segments.
The complementary color filter segments, however, bring about the following problem as to the interpolation of luminance data. Let luminance data calculated by interpolation using correlation and containing high frequency components be referred to as high-frequency luminance data. Even if the colors of a subject to be picked up do not change, the color components of the high-frequency luminance data are rendered differently when transformed to the primary colors R, G and B, depending on the position of calculation. As a result, noise appears in an image in the form of a stripe in the direction in which correlation exists. To reduce this kind of noise, the high-frequency luminance data are subjected to low-pass filtering in the direction in which correlation exists. Low-pass filtering, however, cancels the improvement in resolution.
It is an object of the present invention to provide a solid-state image pickup apparatus realizing both of high sensitivity and high resolution, and a signal processing method therefor.
In accordance with the present invention, a solid-state image pickup apparatus separates the colors of incident light at positions corresponding to the apertures of shield members, which shield the incident light. The color-separated light is transformed to a corresponding signal. The signal is processed to broaden its frequency band. The apparatus includes a solid-state image sensor including a color filter, a sensor array, and a reading circuit. The color filter has color filter segments, which include complementary color filter segments as to a spectral characteristic, for separating the incident light input via the apertures into a plurality of colors each having a particular spectral characteristic. The sensor array has photosensitive cells each for transforming the incident light input via a particular color filter segment to a signal charge. Nearby photosensitive cells are shifted from each other in at least one of the vertical and horizontal directions. The reading circuit sequentially reads out signal charges generated in the photosensitive cells in a preselected order to thereby produce an output signal.
A digitizing circuit digitizes the output signal for thereby outputting pixel data. Assuming actual pixels derived from positions where the photosensitive cells are present and virtual pixels intervening between the actual pixels, a signal processing circuit detects a correlation of the pixel data in the horizontal and/or the vertical direction. The signal processing circuit then multiplies each of the pixel data by a particular constant in accordance with the color of the color filter segment to thereby generate corrected pixel data. Subsequently, the signal processing circuit produces, based on the detected correlation, high-frequency luminance data including high-frequency components for the actual pixels and virtual pixels while producing, based on the pixel data, three primary color data. The signal processing circuit generates, based on such various kinds of pixel data, data relating to luminance and chrominance.
The color filter includes a first group of color filter segments of a first and a second color positioned on the same row and a second group of color filter segments of a third and a fourth color positioned on the same row. The first group of color filter segments and second group of filter segments, which adjoin each other, are shifted from each other by one-half of a pixel pitch. The color filter segments are arranged such that a first sum of products and a second sum of products, which are respectively produced by multiplying the spectral characteristic of the first group by a first and a second constant and multiplying the spectral characteristic of the second group by a third and a fourth constant, are substantially equal to each other, and such that the first and second sums have a relation close to a spectral characteristic representative of luminance.
Also, in accordance with the present invention, a signal processing method begins with a step of separating colors of light incident to a solid-state image sensor having a first group of color filter segments of a first and a second color positioned on the same line and a second group of color filter segments of a third and a fourth color positioned on the same line. The first group of color filter segments and second group of filter segments, which adjoin each other, are shifted from each other by one-half of a pixel pitch, and each corresponds to the aperture of a particular shield members that shield incident light.
Photosensitive cells, each of which adjoins a particular color filter segment, transforms the color-separated light to corresponding signal charges. The signal charges are read out in the form of an analog signal and then processed. The processed analog signal is converted to a digital signal. The digital signal are written, in the form of pixel data, in a memory from which the pixel data can be repeatedly read out a plurality of times. Correlation in at least one of the horizontal and vertical directions is detected by using the pixel data read out of the memory. Each of the pixel data is multiplied by a particular preselected constant assigned to a particular color to thereby generate corrected pixel data. Calculation, which gives consideration to the direction of the correlation detected, is executed with the corrected pixel data to thereby generate high-frequency luminance data containing high-frequency components. Three primary color data are generated by using the pixel data read out of the memory. Luminance data and chrominance data are produced by using the high-frequency luminance data and the three primary color data.
More specifically, signals derived from the photosensitive cells of the first color and the signals derived from the photosensitive cells of the second color are multiplied by a first and a second constant, respectively. The resulting products are added to produce a first sum. Signals derived from the photosensitive cells of the third color and the signals derived from the photosensitive cells of the fourth color are multiplied by a third and a fourth constant, respectively. The resulting products are also added to produce a second sum. The first to fourth constants are selected such that the first and said second sums have a relation close to a spectral characteristic representative of luminance.